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Abstract 

The principle underlying this paper is the basic observation that the problem of simultaneously 
solving a large class of composite monotone inclusions and their duals can be reduced to that of 
finding a zero of the sum of a maximally monotone operator and a linear skew-adjoint operator. An 
algorithmic framework is developed for solving this generic problem in a Hilbert space setting. New 
primal-dual splitting algorithms are derived from this framework for inclusions involving composite 
monotone operators, and convergence results are established. These algorithms draw their simplic- 
ity and efficacy from the fact that they operate in a fully decomposed fashion in the sense that 
the monotone operators and the linear transformations involved are activated separately at each 
iteration. Comparisons with existing methods are made and applications to composite variational 
problems are demonstrated. 
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1 Introduction 



A wide range of problems in areas such as optimization, variational inequalities, partial differential 
equations, mechanics, economics, signal and image processing, or traffic theory can be reduced to 
solving inclusions involving monotone set-valued operators in a Hilbert space H, say 

find x eV. such that z G Mx, (1.1) 

where M : % -> 2 n is monotone and z G H, e.g., [121 [I3l [HI H21 E21 [25l [33l EH EH [41]. In many 
formulations of this type, the operator M can be expressed as the sum of two monotone operators, 
one of which is the composition of a monotone operator with a linear transformation and its adjoint. 
In such situations, it is often desirable to also solve an associated dual inclusion [U El HI El Ell 123 
[271 1281 1501 15T1 152] . The present paper is concerned with the numerical solution of such composite 
inclusion problems in duality. More formally, the basic problem we consider is the following. 

Problem 1.1 Let T~L and Q be two real Hilbert spaces, let A: T~L — > 2^ and B: Q — > iP be maximally 
monotone, let L: % — >■ Q be linear and bounded, let z G %, and let r G Q. The problem is to solve 
the primal inclusion 

find x G U such that z G Ax + L*B(Lx — r) (1.2) 
together with the dual inclusion 

find v G G such that - r G -LA~\z - L*v) + B~ x v . (1.3) 

The set of solutions to (jl.2p is denoted by V and the set of solutions to (|1.3p by V. 

A classical instance of the duality scheme described in Problem 11.11 is the Fenchel-Rockafellar 
framework |32j which, under suitable constraint qualification, corresponds to letting A and B be 
subdifferentials of proper lower semicontinuous convex functions f:T-L—> ]— oo,+oo] and g: Q —> 
]— oo,+oo], respectively. In this scenario, the problems in duality are 

minimize f(x) + g(Lx — r) — (x \ z) (1-4) 

and 

minimize f*(z — L*v) + g*(v) + (v I r) . (1-5) 

Extensions of the Fenchel-Rockafellar framework to variational inequalities were considered in [H \TT\ 
EH [27], while extensions to saddle function problems were proposed in |24j . On the other hand, general 
monotone operators were investigated in [51 U[ E6] in the case when Q = % and L = Id. The general 
duality setting described in Problem 11.11 appears in [l"5 l l28 l 150]. 

Our objective is to devise an algorithm which solves (II. 2D and (jl.3p simultaneously, and which uses 
the operators A, B, and L separately. In the literature, several splitting algorithms are available for 
solving the primal problem (|1.2p . but they are restricted by stringent hypotheses. Let us set 

An H -> 2 n : x ^ -z + Ax and A 2 : U -> 2 n : x ^ L*B(Lx - r), (1.6) 

and observe that solving (|1.2[) is equivalent to finding a zero of A% + A 2 - If B is single- valued and 
cocoercive (its inverse is strongly monotone), then so is A 2 , and (|1.2p can be solved by the forward- 
backward algorithm |10[ [25l 138] . If B is merely Lipschitzian, or even just continuous, so is A 2 , and 
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(jl.2p can then be solved via the algorithm proposed in [39]. These algorithms employ the resolvent of 
A±, which is easily derived from that of A, and explicit applications of A2, i.e., of B and L. They are 
however limited in scope by the fact that B must be single-valued and smooth. The main splitting 
algorithm to find a zero of A\ + A2 when both operators are set-valued is the Douglas-Rachford 
algorithm 114 1 [23| 136j. This algorithm requires that both operators be maximally monotone and 
that their resolvents be computable to within some quantifiable error. Unfortunately, these conditions 
are seldom met in the present setting since A2 may not be maximally monotone |281 [30] and, more 
importantly, since there is no convenient rule to compute the resolvent of A2 in terms of L and the 
resolvent of B unless stringent conditions are imposed on L (see (6j Proposition 23.23] and [19] ). 

Our approach is motivated by the classical Kuhn- Tucker theory [35], which asserts that points 
x G H and v G Q satisfying the conditions 

(0,0) G ( - z + df(x) + L*v, r + dg*{v) - Lx) (1.7) 

are solutions to (jl.4p and (jl.5p , respectively. By analogy, it is natural to consider the following problem 
in conjunction with Problem ll.il 

Problem 1.2 In the setting of Problem ll.il let /C = H © G and set 

M: 7C -»■ 2 K : (x,v) H> (-z + Ax) X (r + B -1 v) and S: K -> K: (x, v) h-> (L*v, -Lx). (1.8) 
The problem is to 

find x G K such that G Mx + Sx. (1.9) 

The investigation of this companion problem may have various purposes [U [151 [281 [30] . Ours is to 
exploit its simple structure to derive a new splitting algorithm to solve efficiently Problem 11.11 The 
crux of our approach is the simple observation that (jl.9p reduces the original primal- dual problem 
(jl.2p - (jl.3p to that of finding a zero of the sum of a maximally monotone operator M and a bounded 
linear skew-adjoint transformation S. In Section[2]we establish the convergence of an inexact splitting 
algorithm proposed in its original form in [39J. Each iteration of this forward-backward-forward 
scheme performs successively an explicit step on S, an implicit step on M, and another explicit step 
on S. We then review the tight connections existing between Problem 11.11 and Problem 11.21 and, in 
particular, the fact that solving the latter provides a solution to the former. In Section El we apply 
the forward-backward-forward algorithm to the monotone+skew Problem 11.21 and obtain a new type 
of splitting algorithm for solving (11.2p and (11.3P simultaneously. The main feature of this scheme, 
that distinguishes it from existing techniques, is that at each iteration it employs the operators A, B, 
and L separately without requiring any additional assumption to those stated above except, naturally, 
existence of solutions. Using a product space technique, we then obtain a parallel splitting method 
for solving the m-term inclusion 

m 

find x G H such that L*Bi(LiX - r»), (1.10) 

i=i 

where each maximally monotone operator Bi acts on a Hilbert space Gi, r% G Gi, and Li : % —> Gi 
is linear and bounded. Applications to variational problems are discussed in Section HI where we 
provide a proximal splitting scheme for solving the primal dual problem (jl.4p - (jl.5p . as well as one for 
minimizing the sum of m composite functions. 
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Notation. We denote the scalar products of 7i and Q by (• j •} and the associated norms by || • ||. 
23 (%, G) is the space of bounded linear operators from T~L to Q, 23 (H) = 23 {%, H), and the symbols — 
and denote respectively weak and strong convergence. Moreover, Ti@Q denotes the Hilbert direct 
sum of % and Q. The projector onto a nonempty closed convex set C C % is denoted by Pc, and its 
normal cone operator by Nc, i.e., 

Afc:*^:^/^* 1 ^' to-*!")*"}. (1 .n) 
|0, otherwise. 

Let M: T~L — > 2^ be a set-valued operator. We denote by ran M = {u € % | (3x G %) u G Mx} the 
range of M, by domM = jx G % \ Mx ^0} its domain, by zer M = {x G U | G Mx} its set of 
zeros, by graM = {(x, u) G % x % [ u G Mx} its graph, and by M _1 its inverse, i.e., the operator 
with graph {(u,x) eHxW | u S Mx}. The resolvent of M is Jm = (Id+M) -1 . Moreover, M is 
monotone if 

(V(x,y) G H x H)(y(u,v) G Mx x My) (x - y | u - u) > 0, (1.12) 

and maximally so if there exists no monotone operator M : T~L — >■ 2^ such that gra M C gra M 7^ gra M. 
In this case, Jm is a nonexpansive operator defined everywhere in "H. For background on convex 
analysis and monotone operator theory, the reader is referred to [6J HQ] . 



2 Preliminary results 

2.1 Technical facts 

The following results will be needed subsequently. 

Lemma 2.1 [9, Lemma 3.1] Let (a n ) n£ N be a sequence in [0, +00 [, let (/3 n ) ne N be a sequence in 
[0, +00 [, and let (e n ) n< =n be a summable sequence in [0, +00 [ such that (Vn G N) a n+ \ < a n — j3 n + e n . 
Then (a n ) nS N converges and (/3 n ) ng N is summable. 

Lemma 2.2 [9~1 Theorem 3.8] Let C be a nonempty subset of H and let (x n ) ng N be a sequence in %. 
Suppose that, for every x G C, there exists a summable sequence (£ n )neN ^ n [0, +oo[ such that 

(Vn G N) ||x n+ i - x|| 2 < \\x n - xf + e n , (2.1) 

and that every sequential weak cluster point of (x n ) n( =N is in C. Then (x ra ) n( =N converges weakly to a 
point in C. 

Definition 2.3 [2, Definition 2.3] An operator M : % — > 2^ is demiregular at x G domM if, for 

every sequence ((x n ,u n )) n< =n in graM and every u G Mx such that x n — x and u n — > u, we have 

Lemma 2.4 [21 Proposition 2.4] Lei M: % — >• 2^ anrf Zei x G domM. Then M is demiregular at x 
in each of the following cases. 

(i) M is uniformly monotone at x, i.e., there exists an increasing function (p: [0, +oo[ — > [0, +00] 
that vanishes only at such that (Vw G Mx)(y(y,v) G graM) (x — y \ u — v) > </>(||x — y\\). In 
particular, M is uniformly monotone, i.e., these inequalities hold for every x G domM and, a 
fortiori, M is a-strongly monotone, i.e., M — aid is monotone for some a G ]0, +oo[. 
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(ii) Jm is compact, i.e., for every bounded setC cH, the closure of Jm{C) is compact. In particular, 
domM is boundedly relatively compact, i.e., the intersection of its closure with every closed ball 
is compact. 

(iii) M : T-L — > H is single-valued with a single-valued continuous inverse. 

(iv) M is single-valued on domM and Id— M, i.e., for every bounded sequence (x n ) ng N in domM 
such that (Mx n ) ng N converges strongly, (x n ) n <=N admits a strong cluster point. 



2.2 An inexact forward-backward- forward algorithm 

Our algorithmic framework will hinge on the following splitting algorithm, which was proposed in the 
error- free case in |39j . We provide an analysis of the asymptotic behavior of a inexact version of this 
method which is of interest in its own right. 

Theorem 2.5 Let H be a real Hilbert space, let A: H — > 2^ be maximally monotone, and let 
B: H. — > "K be monotone. Suppose that zer(A + B) ^ and that B is (3-Lipschitzian for some 
/3 £ ]0, +oo[. Let (a n ) n gN> (^n)ngN? and (en)neN be sequences in H such that 



||a n || < +oo, ||^n|| < +oo, and ||e n || < +oo, 

nGN n£N nGN 

let xq € H, let e £ ]0, l/(/3 + 1)[, let (7 n )neN be a sequence in [s, (1 — e)//3], and set 



(Vn G N) 



y n = x n - j n (Bx n + a n 

Pn = J -y n A V n + h n 
Qn=Pn- ln{Bp n + C n ) 
x n+l — ""n Vn Qn' 



Then the following hold for some x £ zer(A + B). 

|| 2 < +oo and J] neN \\y n - q n || 2 < +oo. 

(ii) x n — 1 x and p n — 1 x. 

(iii) Suppose that one of the following is satisfied. 

(a) A-\- B is demiregular at x. 

(b) A or B is uniformly monotone at x. 

(c) intzer(A + B) ^ 0. 

TTien x n — > a: and p n — )■ If 7 . 
Proof. Let us set 



(2.2) 



(2.3) 



(Vn G N) 



y n — x n ^f n BXf 

Pn = J-y n A Vn 
{Qn =Pn~ lnBp n 



and 



U r 



In \ X r- 



Pn) + B Pn ~ Bx n 



y n -(ln-yn + In- 



(2.4) 



5 



Then 



(Vn G N) y n -p n e j n Ap n and u n = 7n (y n - p n ) + Sp n G Ap n + Bp 



(2.5) 



Now let x G zer(A + £?) and let n G N. We first note that (a;, —j n Bx) G gra7 n A. On the other hand, 
([23]) yields (p n ,y n -p n ) G gra7„A. Hence, by monotonicity of -y n A, {p n - x \ p n -y n - j n Bx) < 0. 
However, by monotonicity of B, (p n — x \ j n Bx — j n Bp n ) < 0. Upon adding these two inequalities, 
we obtain (p n — x \p n — y n — j n Bp n ) < 0. In turn, we derive from (|2.4p that 

27™ (P n - x I Bx„ - Bp n ) =2{p n - x \p n -y n - -y n Bp n ) + 2 (p n - x \ j n Bx n + y n - p n ) 

<2{p n -x\ j n Bx n + y n - p n ) 
= 2{p n -x\x n - p n ) 



x t 



X 



X 



Pn 



(2.6) 



and, therefore, using the Lipschitz continuity of B, that 

|2 



X r 



Vn + Qr 



x 





\(Pn 




\Pn " 


< 


1 


< 


|f n 


< 


I <En 



(p n -x)+ j n {Bx n - Bp, 

l|2 ><■,„, l~ „ 1 



Pnli 2 +7^ll^n-Sp n || 2 



ac 1 1 — 1 1 a? 

^H 2 -(l-7'/3 2 )ll^n-p„|| 2 
2 



a; 



2ii _~ i|2 



We also derive from (|2.3p and (|2.4p the following inequalities. First, 



IVn-Vnl 



7n||On|| < \\a n 



(2.7) 



(2.8) 



Hence, since J 7 „a is nonexpansive, 



In turn, we get 



Pn\\ = \\J ln Ay n - J ~fnAy n ~ M 

< \\J ln AV n ~ J l n Ay n \\ + ll b n| 

< ||Wn - Vn\\ + W b n\\ 

< \\an\\/P+ \\b n \\. 



Qn\\ = \\Pn ~ lnBp n ~ P n + 7n(Bp n + C n )\\ 

< \\Pn ~ Pn\\ + ln\\Bp n ~ B P n \\ + ln\\c n 

< (1 +7n/3)||Pn ~Pn\\ +7n||c„|| 

< 2(||a n ||//3 + ||6 n ||) + WcnW/p. 



(2.9) 



(2.10) 

Combining ([MD, (ZED, and ^M) yields ||e n || < \\y n - yj + \\q n - qj < 3\\a n \\ / f3 + 2\\b n \\ + \\c n \\ / f3 
and, in view of ()2.2|) . it follows that 

^||e fc ||<+oo. (2.11) 

fcGN 



Furthermore, (H3]), (J21J, and ([12]) imply that 

ll^n+l f|| — H^-n 2/n ~l~ 9n ^11 — ll^ra ~l~ 9n ""II ll^^ll — W^n ^\\ \\^n\ 



(2.12) 
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Thus, it follows from (|2.1ip and Lemma |2. II that (x^keN is bounded, and we deduce from (j2.4j) that, 
since the operators B and (J 7fc A)fceN are Lipschitzian, (yk)keN, (Pfc)fceNj and (qk)k£N are bounded. 
Consequently, /i = sup fcgN \\xk — y k + q k — x\\ < +00 and, using (|2.3j) . ()2.4j) . and (|2.7p . we obtain 



x 



n+l 



X 



— \\ x n 

— ||«"n 

— \\ x n 

— II 



Vn + Qn 



X 



y n + Qn- x + e n\ 



Vn + Qn 

2 



X 



+ 2(x n -y n + q n 



x 



+ er 



X 



E 1 1 X r 



Pn\\ + £ r 



where 



2yu||e n || + ||e r 



(2.13) 



It follows from (f2TTTT) . (i2TT3l) . and Lemma O that 

\ " 11 ~ i|2 



neN 



(2.14) 



Hence, since (12.2p and (12. 9j) imply that SngN ||p n — p n || < +00, we have J2nen llPn ~~ Pnll 2 < + 
We therefore infer that XlneN \\ x n ~ Pn\\ 2 < +00. Furthermore, since fj2.4|) yields 

(VneN) \\y n 



00. 



9 n | 



|j/n *?n + e n|| 

\x n - p n - 7n(-B« n - BpJ + e n || 2 

2 o2 



<3(||aB„-p n || +7n/3 Ikn-Pnll + ll e " 

< 6||x n -p n \\ 2 + 3||e n || 2 , 



(2.15) 



we derive from (|2.1ip that X^neN \\Vn ~ Qn\\ 2 < +00. 



that 



(2.16) 



It follows from (|2,14p . the Lipschitz continuity of B, and (j2~^ 

£?p n — Bx n — > and u n — ► 0. 

Now, let w be a weak sequential cluster point of (x n ) n ^, say x kn w. It follows from (|2.5p that 
(Pfc n , Ufc n )nGN lies in gra(A + B), and from (12.141) and (I2.16P that 

(2.17) 



w and life 



0. 



Since -B : % — > H is monotone and continuous, it is maximally monotone [6l Example 20.29]. Further- 
more, since domB = H, A + B is maximally monotone [6j Corollary 24.4(i)] and its graph is therefore 
sequentially closed in W eak x U strong Proposition 20.33(h)]. Therefore, (to } 0) G gra(A + ,B). Us- 
ing (I2.13p . (12. 1 1 H . and Lemma 12.21 we conclude that there exists x G zer(A + B) such that x n x. 
Finally, in view of (i) p n — x. 



(hi) (a) : As shown in (ii) p n — a;. In turn, it follows from (12.2p that p n = p n + b n — x. Moreover, 
(12461) yields u n ->■ and {23} yields (Vn € N) (p n ,w n ) G gra(A + B). Altogether, Definition O 
implies that p n ^tx and, therefore, that p n = p n — b n — > x. Finally, it results from (i) that x n — > x. 



(iii)(b) (iii)(a) : The assumptions imply that A + B is uniformly monotone at x. Hence, the result 



follows from Lemma l2.4fi 



(iii)(c) It follows from (|2X3jh (l2~TTD . (ii) , and [9, Proposition 3.10] that In turn, [(i)] yields 



Pn 



X. □ 



Remark 2.6 The sequence (a n ) ne N, (b n ) n( z^, and (c n ) ng N in (|2.3|) model errors in the implementation 
of the operators. In the error-free setting, the weak convergence of (cc n ) n( =N to a zero of A + B in 
Theorem 12. qYii) | follows from [39, Theorem 3.4(b)]. 
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2.3 The monotone+skew model 



Let us start with some elementary facts about the operators M and S appearing in Problem 11.21 
Proposition 2.7 Consider the setting of Problem \l.l\ and Problem \1.2l Then the following hold. 

(i) M is maximally monotone. 

(ii) SeS (K), S* = -S, and \\S\\ = \\L\\. 
(hi) M + S is maximally monotone. 

(iv) (V 7 G ]0, +oo[)(Vx G H)(Vv G Q) J lM {x, v) = ( J lA {x + jz) , J lB -i (v - 7 r)) . 

(v) (V 7 G ]0,+oo[)(Vx G U)(Vv G Q) 

J-y S (x,v)= ((Id + 7 2 L*L)- 1 (x- 7 L*y), (Id + 7 2 LL*)- 1 (y + 7 Lx)). 



Proof, (i) Since A and -B are maximally monotone, it follows from [61 Propositions 20.22 and 20.23] 
that A x B^ 1 is likewise. In turn, M is maximally monotone. 

(ii) The first two assertions are clear. Now let (x,v) G /C. Then ||S(x, i;)|| 2 = ||(L*?;, — Lx)|| 2 = 
\\L*vp + ||Lx|| 2 < ||L|| 2 (||w|| 2 + ||x|| 2 ) = ||L|| 2 ||(x,?;)|| 2 . Thus, ||5|| < Conversely, ||x|| < 1 => 

||(x,0)|| < 1 \\Lx\\ = ||5(x,0)|| < ||S||. Hence \\L\\ < \\S\\. 



(hi) : By (i) , M is maximally monotone. On the other hand, it follows from (ii) that S is monotone 
and continuous, hence maximally monotone [6, Example 20.29]. Altogether, since domS = JC, it 
follows from [6, Corollary 24.4] that M + S is maximally monotone. 



(iv) This follows from [BJ Proposition 23.16]. 



(v) : Let (x, v) G K, and set (p, q) = J 7 s(x, v). Then (x, v) = (p, q)+jS(p, q) and hence x = p+jL*q 
and v = q — jLp. Hence, Lx = Lp + jLL*q and L*v = L*q — jL*Lp. Thus, x = p + 7 L*t> + j 2 L*Lp 
and therefore p = (Id + 7 2 L*L) _1 (x — jL*v). Likewise, v = q — jLx + 7 2 LL*g, and therefore q = 
(Id + 7 2 LL*)- 1 (v + 7Lx). □ 

The next proposition makes the tight interplay between Problem 11.11 and Problem 11.21 explicit. 
An alternate proof of the equivalence (hi) ^ j(iv) ^(v)| can be found in [28] (see also [31 [151 [23 [30] for 



partial results); we provide a direct argument for completeness. 
Proposition 2.8 Consider the setting of Problem \l.l\ and Problem H.IA Then 

(i) zer(iW + S) is a closed convex subset ofVxD. 
Furthermore, the following are equivalent. 

(ii) z G ran(^4 + L* o B o (L • -r)). 
(hi) V 0. 

(iv) zer(M + 5) ^ 0. 
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(v) 

(vi) -r G ran(-L o A" 1 a (z - L*-) + B' 1 ) 



Proof. The equivalences I(ii)k4](iii) and (v)^(vi) are clear. Now let (x,v) G /C. 



We derive from from (jl.8p that (x, v) 6 zer(M + 5) O G -z+Ax + L*-u and 6 r + L? 1 v-Lx 



(z — L*v G Ax and Lx - r e B l v) (z — L*t; G Ax and w G L?(Lx — r)) = 
and L*v G L*(B(Lx - r))) 4 z £ ii + L*(B(Lx - r)) <^> x G P. Similarly, (z 



(z - L*v G Ax 
L*v G Ax and 



Lx 
r — 



r G B l v) <=> (x G A -1 (z - L*u) and r - Lx G — B" 



(Lx G L(A~ x (z - L*v)) and 



Lx G =4> r G L(A 1 (z - L*u)) - L? ^ u G V. Finally, since M + S is maximally 



monotone by Proposition I2.7|(iii) zer (iW + S) is closed and convex (6J Proposition 23.39]. 

(iii)| =» ](iv)| In view of (fL8l) . i £ P <s> z 6 Ax + L*(B(Lx - r)) (z-L*w G Ax and w G 

L?(Lx - r)) ((3 w £ Q) z £ Ax + L*u; and - r G L? _1 



it; 



Lx) ^>(3wG6) (x, w) G zer(M + 5). 



IV 



(hi) and (iv 



v) These follow from 
- L*v) - 



v)L» ](iv)| v G L> <=> r G LA _1 (z - L*t>) - L?" 1 ?; (3 y G (y G A _1 (z - L*w) and r G Ly 
B^w) <^ (3 y G U) (0 G -z + Ay + L*u and G r + - Ly) ^ (3 y G (y, u) G zer(M + 5). □ 



Remark 2.9 Suppose that z G ran(A + L*B(L ■ — r)). Then Proposition 12.81 assert that solutions to 
(|1.2p and (|1.3p can be found as zeros of M + S. In principle, this can be achieved via the Douglas- 
Rachford algorithm applied to fj 1 .91) : let (a n ) ng N and (b n ) ne pj be sequences in /C, let (X n )n<=N be a 
sequence in ]0,2[ such that b n — 0, I] neN A n (|ja n || + ||6 n ||) < +oo, and XlngN An(2 ~~ A n ) = +°°' ^ et 
y S /C, let 7 G ]0, +oo[, and set 



(Vn G 



y n+1 = y n + A n (J 7 M(2« n - y„) + a„ - a; T 



(2.18) 



Then it follows from Proposition 12 . 7[(iJ ^(iii) | and Theorem 2.1(i)(c)] that (x n ) n ^ converges weakly 



to a point in zer(iW + S). Now set (Vn G 
t> n = (&i,n>&2,n)- Then, using Proposition 12. 7pTvJfe|(v)[ (|2.18p becomes 



(x n ,v n ), y n = (yi,„,y 2 ,n), 



(Vn G 



x n = (Id + 7 2 L*L)- 1 (yi jn - jL*y 2 , n ) + h,n 
v n = (Id + 7 2 LL*)~ 1 (y 2 ,n + ^Ly^ n ) + 6 2 ,„ 

= 2/i,n + A„(J 7 a(2x„ - yi jn + 72) + a^ n - x n ) 

Z/2,ra+l = U2,n + A„ ( J 7 £-l (2u n - y 2 ,n ~ 7 r ) + a 2,n ~ «n) • 



and 



(2.19) 



Moreover, (x n ) n( =N converges weakly to a solution x to (II. 2D and (t>„)neN to a solution U to (11.31) such 
that z — L*v G Ax and U G B(Lx — r). However, a practical limitation of ([2.19P is that it necessitates 
the inversion of two operators at each iteration, which may be quite demanding numerically. 



Remark 2.10 It follows from (|2.12p that the error-free version of the forward-backward-forward 
algorithm (|2.3p is Fejer- monotone with respect to zer (A + B), i.e., for every n G N and every x G 
zer(A + B), ||a; n +i — x\\ < \\x n — x\\. Now let n G N. Then it follows from [5j Section 2] that there 
exist A n G [0, 2] and a closed affine halfspace H n C H containing zer(A + B) such that 

x n+1 = x n + X n (P Hn x n - x n ). (2.20) 

In the setting of Problem 11.21 H n and \ n can be determined easily. To see this, consider Theorem 12.51 
with T-L = K, A = M, and B = S. Let x G zer(iVL + S) and suppose that q n ^ y n (otherwise, 
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we trivially have H n = 7C). In view of (|2.3p . y n — p n S r y n Mp n and —j n Sx G j n Mx. Hence, 
using the monotonicity of j n M and Proposition I2.7[(ii)| we get < (p n — x\y n —p n + -y n Sx) = 
(Pn I Vn ~ Pn) ~ ( x I Vn ~ Pn) + In (S*p n \ x) = (p n \ y n - p n ) - (x \ y„ - p„ + j n Sp n ). Therefore, 
we deduce from (12. 3D that (x \ y n — q n ) < (p n \ y n — p n ) = (p n \ y n — q n ). Now set 

H n = {xeK\(x\y n -q n )<(p n \y n -q n )} and X n = 1 + 7 2 n Pn ~ . (2.21) 

WPn x n\\ 

Then zer(M + S) C H n and \ n < 1 + 7^,||5'|| 2 < 2. Altogether, it follows from ((23} and the 
skew-adjointness of S that 

X n + K(P Hn X n ~ X n ) =X n + \ n ( (Pn- *n\j n -q n ) \ ( ^ _ gJ 

V 1 1 Vn Qn 1 1 / 



i x / (Pn ~X n \x n -p n + J n S(p n - X n )) i 

n+ n{ "x n -p n + ln S(p n -x n W ]{Vn 

2 



\ I II W *P^Tl 1 1 1 

= ^ + " {\\*n-Pn\\ 2 +ri\\S(p n -X n W) ^ ~ VJ 

= x n - y n + q n = x n+1 . (2.22) 

Thus, the updating rule of algorithm of Theorem 12.51 applied to M and S is given by (|2.20p - (|2.2ip . In 
turn, using results from [5], this iteration process can easily be modified to become strongly convergent. 



3 Main results 

The main result of the paper can now be presented. It consists of an application of Theorem 
to find solutions to Problem 11.21 and thus obtain solutions to Problem 11.11 The resulting algorithm 
employs the operators A, B, and L separately. Moreover, the operators A and B can be activated in 
parallel and all the steps involving L are explicit. 

Theorem 3.1 In Problem \l.ll suppose that L ^ and that z £ ran (A + L* o B o (L ■ — r)). Let 

(oi,n)n€N» (&l,n)neNj an d (ci, n )nGN be absolutely summable sequences in%, and let (a 2 , n )neN, (^2,n)neN, 
and (c2, n )nGN be absolutely summable sequences in Q. Furthermore, let xo £ H, let vo E Q, let 
e € ]0, 1/(||L|| + 1)[, let (7 n )neN be a sequence in [e, (1 — ], and set 



(Vn G 



•En 7n 

(L*v n + ai 
v n + 7 n (Lx 

n "r a 2,n) 



(3.1) 



2/l,n 

.'/•>.„ 

Pl,n = Jj n A(Ul,n + InZ) + &l, n 
P2,n = J ln B- 1 (U2,n ~ 7n»~) + &2,n 
gi,n = Pl,n - ln{L*P2,n + Ci, n ) 
92,™ = P2,n + 7n(ipi,n + C 2 ,n) 
^n+l = Sn - 2/l,n + Ql,n 
V n +1 =V n - y 2 ,n + Q2,n- 

Then the following hold for some solution x to (11.21) and some solution v to (11.31) such that z—L*v £ Ax 
and v £ B(Lx — r). 



(i) x n - p ltn -> and w n - p 2 , r 



0. 
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(ii) x n x, p 1>n -± x, v n -± v, and p 2 , n v. 

(iii) Suppose that A is uniformly monotone at x. Then x n — > x and pi <n —> x. 

(iv) Suppose that B^ 1 is uniformly monotone at v. Then v n — > v and p 2<n v. 

Proof. Consider the setting of Problem 11.21 As seen in Proposition 12.71 M is maximally monotone, 
and S G ¥> (JC) is monotone and Lipschitzian with constant ||L||. Moreover, Proposition 12.81 yields 



/ zer(M |S)cPxP. 



(3.2) 



Now set 



(Vn G 





— (^n j Vn) 










Vn 


= (yi,n,V2,n) 






f 

d n — 


(«l,n, G2,n 


< Vn 


= (Pl,n,P2,n) 


and 


< 


K = 


(&l,ri! &2,n) 


Vn 


= (Pl,n,P2,n) 








(ci,n,C 2 ,n) 


,9n 


= \Ql,n, Q2,n)- 











(3.3) 



Then, using (fL8|) and Proposition |2JHiv)t (IBTTj) we can written as (|2.3p in /C. Moreover, our assump- 

(3.4) 



tions imply that (|2.2p is satisfied. Hence, using (|2.9p . we obtain 

Pi,n ~ Pi,n -> and p 2 ,n - p 2i „ -> 0. 
Furthermore, we derive from (|2.4p and (jl.8p that 

a, J 7n 1 ( a; n - Pl,n) + L*p 2n ~ L*V n G Ap Xn - Z + £*P2,n 

(Vn G N) < _ _ _ _ 

[in { v n ~ P2,n) ~ Lp lj7l + Lx n G B L p 2>n + T~ Lp X ,n- 

These observations allow us to establish the following. 



(3.5) 



(i) &(ii) These follows from Theorem 12. 5|(i)| & (ii) applied to M and S in /C. 
(iii) Since x solves (|1.2p . there exist u G T~i and v € G such that 

u £ Ax, v€B(Lx — r), and z = u + 

Now let n G N. We derive from (|3.5p that 

7n 1 ( x n ~ Pl,n) ~ L*V n + Z G 4Pl,n and 7^ X (% - P2,n) + ~ r G B~ X p 2 ^ 

which yields 

7n ~~ Pl,n) — L*V n |Z€ ^Pl,n and P2,n £ B(y~ 1 (v n —P2,n) + Lx n — r). 

Now set 



(3.6) 



(3.7) 



(3.8) 



-pi,n||(e ||Pl,n - ^11 + ll^ll ||«r. 



and /3 n = e - p 2 ,n\\ ||P2,-. 



(3.9) 
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It follows from (|3.6p , (|3.8p , and the uniform monotonicity of A that there exists an increasing function 
4>: [0, +00 [ — > [0, +00] that vanishes only at such that 



a n + (x n - x \ L*v - L*v n ) 



> e 



'l.n X\\ \\Xr, 



^ \\Pl,n X\\ \\Xr. 



■ Pl,n\\ + (Pl,n ~ x n I L*v - L*v n ) + (x n - X \ L*v - L*v n ) 

■Pl,n\\ + {Pl,n -X I L*V - L*V n ) 



> (Pl,n ~ X I l n l {x n - Pl >n ) - L*V n + L*v) 



= (Pl,n ~ X j Tn 1 ^ 
> (p(\\pi,n -X\\)- 



Pi,'. 



L*v n + z - u) 



(3.10) 



On the other hand, since B is monotone, (|3.9p . (|3.6p . and ()3.8[) yield 

Pn + {x n ~x \ L*p 2 , n ~ L*v) > {^{Vn - p 2 ,n) + L(x n - x) \ p 2 , n ~ v) 

= ((in 1 ^ ~P2,n) + Lx n - r) - (Lx-r) j p 2 ,n ~ v) 

> 0. 

Upon adding these two inequalities, we obtain 

On + Pn + \\x n ~ x\\ \\L\\ \\P2,n -V n \\ > «n + (3 n + (x n - X j L*(p 2 ,n ~ «n)) > <KI|Pl,n ~ «|| 



(3.11) 



(3.12) 



Hence, since |(ii)| [(i)j and (pTl"]) imply that the sequences (x n ) neN , (t>n)neN, (pi,n)neN, and (p2,n)neN 
are bounded, it follows from (13. 9p . (13. 4p . and (i) that ^(||pi,n — x||) — > 0, from which we infer that 
P\,n — > x and, by (|3.4p . that pi 5 „ — > x. In turn, (i) yields x n — > x. 



IV 



Proceed as in (iii) , using the dual objects. □ 



Remark 3.2 Using a well-known resolvent identity, the computation of p2, n m (13. ip can be performed 
in terms of the resolvent of B via the identity J lnB -iy = y — InJ^-^si^n 1 y)- 

Remark 3.3 Set Z = {(x,v) G V x V \ z - L*v G Ax and v G B(Lx-r)}. Since Theorem 13.11 is 
an application of Theorem [23] in /C, we deduce from Remark [2 . 1 1 that the updating process for (x n , v n ) 
in (13.ip results from a relaxed projection onto a closed affine halfspace H n containing 2, namely 



(x n+ i,V n+ i) = (x n ,V n ) + X n (P Hn (x n ,V n ) - (x n ,V n )), 



(3.13) 



where 



H n = {(x,v) G K I (x j y 1>n - qi :n ) + (v \ y 2 , n ~ qz,n) < (Pl.n I 2/l,n ~ ?l,n) + (P2,n I 2/2,n - g2,n)} 

2 ||-C(pi,n - a:n)|| 2 + ||^*(P2,n - ^„)|| 2 



and A T 



1 + 7* 



||pi,rc - X n \\ 2 + ||p 2 ,n - Vnl 



(3.14) 



In the special case when Q = 7~L and L = Id, an analysis of such outer projection methods is provided 
in m\. 



Corollary 3.4 Let A\ : H — > 2^ and A2'. 7i — > 2^ be maximally monotone operators such that 
zer(^4i + A2) 7^ 0. Let (&i,n)neN & n d (&2,n)neN be absolutely summable sequences in H, let xq and vq 
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be in %, let e G ]0, l/2[, iei (7 n )neN fre a sequence in [e, 1 — e], and set 



(Vn G N) 



Pl,n = «/ 7n Ai(a:n - Tn^n) + &l,n 
P2,n = J^A- 1 ^ + Tn^n) + fa, 
Xn+l = Pl,n + ln{v n ~ P2,n) 
_ V n+ i = p 2t n + ln(Pl,n ~ %n)- 



ji 



(3.15) 



Then the following hold for some x £ zev(Ai + A2) and some v G zer(— A 1 o (— Id) + A 2 ) smc/i f/iai 
— v G vlix and v G A^x. 

(i) x n — 1 x and v n — v U. 

(ii) Suppose that A\ is uniformly monotone at x. Then 

(iii) Suppose that A^ 1 is uniformly monotone at v. Then v n — > v. 

Proof Apply Theorem O with </ = H, L = Id, A = A x , B = A 2 , r = 0, and z = 0. □ 

Remark 3.5 The most popular algorithm to find a zero of the sum of two maximally monotone 
operators is the Douglas- Rachford algorithm [TTJ [T31 [23j [36] (see (|2.18p ). Corollary 13.41 provides an 
alternative scheme which is also based on evaluations of the resolvents of the two operators. 

Corollary 3.6 In Problem suppose that L / and that zei(L*BL) 7^ 0. Let (oti,n)neN an d 
(ci,n)neN be absolutely summable sequences inTi, and let (a2, n )neN, (frn)ngN; and (c2, n )neN be absolutely 
summable sequences in Q. Let xq G H, let vo G Q, let e G ]0, 1/(||L|| + 1)[, let {^ n )n<m be a sequence 
in [e, (1 — e)/||L|| ], and set 



Then the following hold for some x G zer(L* BL) and some v G (ranL) D B(Lx). 

(i) x n — 1 x and v n — v. 

(ii) Suppose that B~ l is uniformly monotone at v. Then v n — > v. 
Proof. Apply Theorem 13.11 with A = 0, r = 0, and z = 0. □ 

Remark 3.7 In connection with Corollary 13.61 a weakly convergent splitting method was proposed 
in |29] for finding a zero of L*BL. This method requires the additional assumption that ranL be 
closed. In addition, unlike the algorithm described in (13.16p . it requires the exact implementation of 
the generalized inverse of L at each iteration, which is challenging task. 

Next, we extend (|1.2p to the problem of solving an inclusion involving the sum of m composite 
monotone operators. We obtain an algorithm in which the operators (-Bj)i<j< m can be activated in 
parallel, and independently from the transformations (Lj)i<j< m , 



(Vn G N) 



S n = ln{L*V n + ax t n) 

Un = V n + 7n{Lx n + 02, n ) 

Pn = J ln B^Vn + K 

X n+1 = X n - ^ n (L*p n + Ci, n ) 

V n +1 =Pn~ 7n(Ls n + C 2 , n )- 



(3.16) 
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Theorem 3.8 Let z G H and let (wj)i<j< m be reals in ]0, 1] such that Y^iLi^i = 1- F° r every 
i G {1, ...,m}, let (Gi,\\ ■ HeJ be a real Hilbert space, let n G Gi, let Bi\ Gi — > 2 Si be maximally 
monotone, and suppose that ^ Li G S (H,Gi)- Moreover, assume that 



G ran WjL* oB ; o (Lj 



(3.17) 



i=l 



Consider the problem 



find x £ % such that z G ^^WjL*Bj(LjX — r^), 



(3.18) 



i=l 



and the problem 

m 

find v\ G Qi, . . . , v m G £7 m suc/i i/iaf ^^£JiL*Vj = z and (3x € %) < 



i=l 



«l G Bi(Lix - ri) 



S B m {L m x r m ). 



(3.19) 



iVou>, /or every i £ {1,... , m}, Zei (ai,i, n )neN anc ^ ( c l,i,n)neN absolutely summable sequences in H, 
let (a2,i, n )w=N, (bi,n)n<aN, and (c2,j in )nGN &e absolutely summable sequences in Gi, let x^o G and lei 
^i.o ^ Furthermore, set j3 = maxi<j< m ||Lj||, Ze£ £ € ]0, l/(/3 + 1)[, Zei (7 n )neN be a sequence in 
[e, (1 — e)/0\, and set 

•En / ij— \ wiXi t fi 

For i = 1, . . . , m 

yi,i,n — n r )n\^-'i'^i,n Q>l,i } n) 
2/2,i,n = Vi,n + ln(LiX^ n + a 2)i , n ) 

Pl,n = El=l ^iVl,i,n + 7n^ 

For i = 1, . . . , m 

P2,i,n = J Jn Br l (V2,i,n ~ 7n r i) + &i,n 
9l,i,n = Pl,n ~ Jn(L*P2,i,n + Cl,j,n) 
q2,i,n = P2,i,n + Jn(LiPl, n + C 2 ,i, n ) 
3*i,n+l — 3?i,n 2/1, i,n ~t" Ql,i,n 
Vi, n +1 = Vi t n — 2/2,1,71 + 92,*,n- 

T/ien f/ie following hold for some solution x to (|3.18p and some solution («i)i<i<m ^° (13. 19H suc/i f/iai, 
/or every ie{l,... ,m}, Vj G Bj(LjX-rj). 



(Vn G 



(3.20) 



i x r 



x and, /or every i G {1, . . . , m}, vi n — i>j. 



(ii) Suppose that, for every i G {1, ...,m}, B, L 1 is strongly monotone at Uj. Then, for every i G 

{1, . . . ,7Jl}, Vi, n -> 

Proof. Let be the real Hilbert space obtained by endowing the Cartesian product H m with the 
scalar product (• | -) u : (x,y) h-> w t (x« | y*}, where a; = (a?i)i<i< m an d y = (yj)i<i< m denote 

generic elements in The associated norm is || ■ x (-)• y / ^™ =1 Wj||xj|| 2 . Likewise, let G denote 
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the real Hilbert space obtained by endowing Q\ x • • • x Q m with the scalar product and the associated 
norm respectively defined by 



■)g ■ (V, z ) ^ ^Z^i (Vi I z i)g t and \\-\\q-V^ 



i=l 

Define 



\ i= i 



V = {(x,...,x) G H I x en} and j : U -> V : x (->• (a;, . . . , x). (3.22) 
In view of (jl.lip . the normal cone operator of V is 

AV:«^2*:^( V± = <" eW|:S ^"' = >- ' f " 6 V '' (3.23) 
|0, otherwise. 

Now set 

m 

A = N v , B:Q^2 G :y^, XB^, L: ->• £: a; (L^i^m, and r = (ri)i<i< m . (3.24) 

i=l 

It is easily checked that A and B are maximally monotone with resolvents 

(V7 G ]0, +oo[) J 7 a : a; 1— > Pv« = J ( ^ ) and ^B^ 1 : f ^ (^yBr 1 ^) 



Ki<m° 



(3.25) 



Moreover, L £ "B (H, Q) and 

^fl-Hfitfh+IZ?^. (3.26) 

Now, set 

l> = e I j» + - r)} 

[X> = {v G £ I -r G -Li4 -1 (i(2!) - + B^v}. 

Then, for every x G H, 

m 

x solves (pTTgj) 44> WjL* (Bi(LiX - n)) 

i=i 

3(uj)i<j< m G XSj(LiX-ri)j 2; = ^ UjLjVi 
* — ^ i=i 

3(uj)i<j< m G X^(LiX-ri)) ^^(z - L*Vj) = 
i=1 / i=i 

O (3« G B(Lj(x) -r)) - G V r± = Aj(x) 

O j(z) G Aj'(x) + L*B{Lj{x) - r) 

j(x) £V CV. (3.28) 
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Moreover, for every 



v solves (J37T9]) 44> 



m 

£ 

i=l 



L*tij) = and (3x€"H) («i)l<i<m € X B^LiX - n) 



45 
45 
45 
45 
45 



(IxeH) j(z)-L*v£V~ 
(3xeU) j(x)€A- 1 (j(z) 



= Aj(x) and v G B(Lj(x) — r) 
L*v) and Lj(x) — r G £J t> 



(3 a; G V = dom A) cc G A -1 (j(z) - L*t>) 
-r G -LA -1 (j(z) - + B l v 
v G X». 



and .La: — r G S 



(3.29) 



variables, (I3.20P can be rewritten as 



Altogether, solving the inclusion (13.18P in H is equivalent to solving the inclusion j{z) G Ax + 
L* B(Lx—r) in and solving (|3.19p in Q is equivalent to solving — r G B~ x v—LA X (j(z)—L*v\ in C?. 
Next, let us show that the algorithm described in (|3.20|) is a particular case of the algorithm described 
in (|3,1|) in Theorem 13, 1[ To this end define, for every n G N, x n = (xi,n)l<i<TO> v n = (vi, n )i<i<m, 

Vl,n = {yi,i,n)l<i<m, V2,n = (y2,i,n)l<i<m, V\,n = j(Pl,n), P2,n = (P2,i,n)l<i<m, Ql jU = (<7l,i,n) l<i<m , 
Q2,n = {Q2,i,n)l<i<m, Ol.n = (ai,i,n)l<i<m, a 2,n = (a2,t,n)l<i<m> &2,n = (bi,n)l<i<m, C\,n = (ci,i,n)l<i<m, 

and C2 in = {c2,i, n )i<i<m- Then we deduce from (|3.24p . (|3.25p . and (|3.26p that, in terms of these new 



y ln = x n - j n (L*v n + ai )fl ) 

V2,n = v n + 7n(Lx n + 2 ,n) 

Pl.n = J 7nA(yi,„ + 7n^) 

P2,n = J- in B- 1 (V2,n ~ 7n0 + &2,n 

<?l,n = Pl,n - Jn(L*p 2 , n + Ci, n ) 

<?2,n = P2,n + 7n(-&Pl,„ + C 2 ,„) 

s n+l = — Vl,n + Ql,n 

« n+1 = «n - l/2,n + <?2,n- 



(Vn G 



(3.30) 



Moreover, 
(ci, n )n6N) 



< maxi<i< m = /3, and our assumptions imply that the sequences (ai, n )neN) 
(^2,n)ngN) (^2,n ) n <=®, and (c 2 , n )ngN are absolutely summable. Furthermore, (|3.17p and 



(pT2"8l) assert that j(z) G ran(A + L* o B o (L ■ -r)). 



It follows from Theorem I3.1](ii)| that there exists x G T and (vi)i<i< m = v G X> such that 
ji(-z) — -L*U G Ax, v G B(Lx — r), a ra — ^ a, and v n — 1 U. Hence, j{x n ) = PyXn Pv% = 
Since (|3.28p asserts that there exists a solution cc to (|3.18p such that x = j(x), we obtain that 
x n = j~ 1 (Pvx n ) j (x) = x. Altogether, by (|3.29p . for every i G {1, . . . ,m}, Vi- n — 1 TTj, where 
(u»)i<i< m solves (|3.19p . 



(ii) Let (wx,yi) an d ( w 2,y2) m graS . We derive from (|3.24|) that (Vi G {l,...,m}) yi^ G 
u;i 5 j and y 2 ,i £ B^ 1 W2,i- Hence, since the operators (-B 4 rl )i<i< m are strongly monotone, there exist 
constants (/?»)i<i< m in ]0, +oo[ such that (y 1 -y 2 \wi- w 2 ) g = Ym.Li (yi,i ~ U2,i I ~ w 2-i) Qi > 
12iLi u iPi II w l,i~ w 2,i || |. > p||«)i—W2|||, where/) = mini<i< m pi G ]0,+oo[. Theref ore, B 1 is strongly 
monotone and hence uniformly monotone. Thus, the result follows from Theorem [371 pv) □ 
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4 Variational problems 



We apply the results of the previous sections to minimization problems. Let us first recall some 
standard notation and results [UHD]. We denote by Fo(H) the class of lower semicontinuous convex 
functions /: 7i — > ]— oo, +00] such that dom/ = {x G 7i \ f(x) < +00} 7^ 0. Now let / G To('H). The 
conjugate of / is the function /* 6 r'o('H) defined by /*: u (->• sup a:GW ((x | n) — f(x)). Moreover, for 
every x G / + ||x — -|| 2 /2 possesses a unique minimizer, which is denoted by proxy x. Alternatively, 

prox / = (Id+S/r 1 = J s/ , (4.1) 

where df:U^2 H : x 1— > jn € % I (Vy G %) (y — x | n) + /(x) < /(y)} is the subdifferential of /, 
which is a maximally monotone operator. Finally, let C be a convex subset of Ti. The indicator 
function of C is denoted by lc, its support function by oc, and its strong relative interior (the set of 
points in x G C such that the cone generated by — x + C is a closed vector subspace of H) by sriC. 
The following facts will also be required. 

Proposition 4.1 Let f G r (%), Zei g G r (£), let L £ ¥> (7i, G), let z eU, and let r eQ. Then the 
following hold. 

(i) zev(—z + df + L* o (dg) o (L • — r)) C Argmin(/ — (■ | z) + g o (L ■ —r)). 

(ii) zer(r - (L o (df*) o (* - £*•)) + 5 5*) c Argmin(/*(z - L*-) + + (r | •)). 
(hi) Suppose that one of the following is satisfied. 

(a) Argmin(/ + g o (L ■ —r) — (• | z)) 7^ and r G sri(L(dom/) — dom g). 

(b) Argmin(/ + g o (L ■ —r) — (• | z)) C Argmin(/ — (• | z)) PI Argming o (L • — r) 7^ and 
r G sri(ran L — dom 5) . 

(c) f = ic o,nd g = ld, z = 0, where C and D are closed convex subset ofH and Q, respectively, 
such that C fl L _1 (r + L>) 7^ and r G sri(ranL — D). 

Then z G ran(9/ + L* o (<9g) o (L • — r)). 



Proof, (i)^(ii) By [Bl Proposition 16.5(h) and Theorem 16.2], zer(— z + df + L* o (dg) o (L • — r)) C 



zer(9(/ — (• [ z) + g o (L ■ — r))) = Argmin(/ — (• | z) + 5 o (L • — r)). We obtain (ii) similarly. 



hi) (a) By [6j Theorem 16.2 and Theorem 16.37(i)], we have 



7^ Argmin(/ + g o (L • — r) — (• | z}) = zer d(f + g a (L ■ —r) — (■ | z)) 

= zer(-z + df + L* o (%) o (L • -r)). (4.2) 



(iii)(b) Since r G sri(ranL — dom g), using (i) and standard convex analysis, we obtain 



Argmin(/ — (• | z)) n Argmin(g o (L • — r)) = zer(— z + <9/) fl zer <9(g o (L • — r)) 

= zer(— z + 9/) fl zer(L* o (<9g) o (L • — r)) 

C zer(-z + df + L*o (dg) o (L • -r)) 

C Argmin(/ + 5 o (L • -r) - (• | z)). (4.3) 
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Therefore, the hypotheses yield zer(—z + df + L* o (dg) o (L • — r)) = Argmin(/ — (• | z)) n Argmin(<7 o 
(L--r))/0. 

(iii)(c)| Since dom(ic + t D (L ■ -r)) = C n L -1 (r + £>), 

Argmin(t c + i£> o (L ■ -r)) = Argmin t CnL -i (r+jD) 

= cnr 1 (r+B) 

= Argmin lq H Argmin(i£) o (L • — r)) ^ 0. (4-4) 



In view of (ii) applied to / = ic, g = ld, and z = 0, the proof is complete. □ 

Our first result is a new splitting method for the Fenchel-Rockafellar duality framework (jl.4j) — (|1.5|) . 

Proposition 4.2 Let f G r (H), let g G r (£), let L E $ (H, Q), let z EU, and let r EG. Suppose 
that L ^ and that 

zeran(df + L*o(dg)o(L--r)). (4.5) 

Consider the primal problem 



and the dual problem 



minimize f(x) + g(Lx — r) — {x \ z) 



minimize f*(z — L*v) + g*(v) + (v | r) . 



(4.6) 



(4.7) 



(Vn G 



(4.8) 



Le£ (ai jn ) ne N, (bi,n)n<=N, and (ci jn ) rag N fre absolutely summable sequences in %, and let (a2,n)raeN) 
(^2,n)nGN> (c2,n)nGN be absolutely summable sequences in Q . Furthermore, let xq G %, let vq G Q, 
let e G ]0, 1/(||L|| + 1)[, let (7 n )neN be a sequence in [e, (1 — ], and set 

Vl,n = X n - J n (L*V n + a^ n ) 

V2,n = v„ + 7„(Lx n + a 2 ,„) 
p 1)Tl = prox 7n/ (yi jn + -y n z) + 6 lin 

P2,n = PrOX 7n9 , (y 2 ,n - 7n0 + &2,n 
9l,n = Pl.n - 1n(L*P2,n + Ci, n ) 
<?2,n = P2,n + lfn(Lpi,n + C2,n) 

X n +1 = x n - yi >n + gi jn 
L v n+ i = v n - y 2 , n + q%,n- 

Then the following hold for some solution x to (14. 6\\ and some solution v to (|4.7j) such that z — L*v G 
dfix) and v G dg{Lx — r). 

(i) x n - p 1}Jl ->■ and v n - p 2 , n -> 0. 

(ii) x„ x, pi >n w n v, and p 2 , n v. 

(iii) Suppose that f is uniformly convex at x. Then x n 

(iv) Suppose that g* is uniformly convex at v. Then v ri 



x and pi )Tl — > x. 
► v and p2, n — > v. 



Proof. Suppose that A = df and B = dg in Problem 11.11 Then, since A 1 = 3/* and S 1 
we derive from Proposition 



i) that the solutions to (|1.2p and (jl.3p are solutions to (|4.6p and 
(14.7jl . respectively. Moreover, (|4.1j) implies that (|4.8|) is a special case of (13. ip . Finally, the uniform 
convexity of a function ip G Tq{T{) at a point of the domain of dtp implies the uniform monotonicity 
of d<p at that point [40|. Section 3.4]. Altogether, the results follow from Theorem 13.11 □ 



18 



Remark 4.3 Here are some comments on Proposition 14.21 



(i) Sufficient conditions for (I4.5|) to hold are provided in Proposition 14.11 



(ii) As in Remark 13.21 if the proximity operator of g is simpler to implement than that of g* , p2, n in 
can be computed via the identity prox 7n9 , y = y — 7 n prox^-1^7^ 1 ?/). 



(iii) In the special case when H and Q are Euclidean spaces, an alternative primal-dual algorithm 
is proposed in [8], which also uses the proximity operators of / and g, and the operator L in 
separate steps. This method is derived there in the spirit of the proximal [M] and alternating 
direction (see [20] and the references therein) methods of multipliers. 



We now turn our attention to problems involving the sum of m composite functions. 

Proposition 4.4 Let z G H and let (wj)i<i<m be reals in ]0, 1] such that Yli=i w * = !• F° r ever y 
i G {1, . . . ,m}, let (Qi, \\ ■ be a real Hilbert space, let n G Qi, let g^ G To(Qi), and suppose that 
^ L{ £ "B (H, Gi) . Moreover, assume that 



m 



z G ran ^ (%) ° (£j 



Consider the problem 



and the problem 



minimize Wj gi(L{X — r^) — (x | z) 



i=l 



minimize Y^w; + ("i I »";) )■ 



(4.9) 



(4.10) 



(4.11) 



For every i G {1, ...,m}, Zei (ai,i, n )neN and (ci ) j )n ) n£ N absolutely summable sequences in %, let 
(a2,i,n)neN; (^i,n)neN; «^ (c2,i,n)neN Z*e absolutely summable sequences in Qi, let xi<q G H, and let 
v%,o G Qi. Furthermore, set (3 = maxi<j< m \\Li\\, let e G ]0,l/(/3 + 1)[, let (j n )n& be a sequence in 
[e, (1 — s)//3], and set 

v-vm 

•En — Z_/i=l ^i%i,n 

For i = 1, . . . ,m 

yi,i,n — •&i,n r )ni,I J iVi^ n + 01,1,71) 
V2,i,n = Vi,n + 7n(LiXi >n + 02,j,n) 
Pl,n = YT=1 UiUl-i-n + In* 

(Vn G N) Fbr i = l,...,m (4.12) 

P2,i,n = PrOX 7n9 * (y 2 ,j,n - 7n r i) + k,n 
qi,i,n = Pl,n ~ ln{L*P2,i,n + Cl,i,„) 
?2,i,n = f>2,i,n + ln{^iPl,n + C2,i,n) 
■Ei,n+1 *£j,n 2/1, i,n "I - Ql,i,n 
_ Vi,n+1 = Wi,n — 2/2,1,71 + <?2,i,n- 

TTien the following hold for some solution x to (|4.10p and some solution (vi)i<i< m to (|4.1ip such that, 
for every ie {!,... ,m}, Ui G dgi(L,jX - n). 
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(i) x n x and, for every i G {1, . . . , m}, v i:U ->> 

(ii) Suppose that, for every i G {l,...,m}, 5* is strongly convex at Wj. Then, for every i G 
{1, . . . ,m}, u ijTl Uj. 

Proof. Define "H, C/, -L, V, r, and j : TL — > V as in the proof of Theorem 13.81 Moreover, set 
f = iv and g: Q -)• ]-oo,+oo] : y (-> 1 9i(yi). Then, / G r (^), g G r (</), /* = L v x, and 
g*:v4 X^Li w « 9i( v i)- Therefore, (|4.9p is equivalent to 

j{z) G ran(d/ + Z* o (dg) o (i • -r)). (4.13) 

Furthermore, (|4.10p and (|4.11|) are equivalent to 



and 



minimize f(x) + g{Lx — r) — (x \ j{z))^ (4-14) 

minimize f*(j(z) — L*v) + g*(v) + (r \ v) G , (4-15) 
veQ 

respectively. On the other hand since, for every 7 G ]0, +00 [, pro x 7 ^ : x (->■ jQZfcLi WjXj) and prox 7g » = 
(prox 79 *)i<j< m , (|4.12p is a particular case of (|4.8p . Finally, in (ii) g* is strongly, hence uniformly, 



convex at t>. Altogether, the results follow from Proposition 14.21 □ 
Remark 4.5 Suppose that (|4.10p has a solution and that 

(n, . . . ,r m ) G sri{(Lirc -yi,.. .,L m x - y m ) \ x G H, yi G dom#i, . . . , y m G domp m }. (4.16) 
Then, with the notation of the proof of Proposition ^. 4| (I4.16P is equivalent to r G sii(L(V) — dom </) = 



sri(.L(dom/) — domgr). Thus, Proposition |4T1 Urn) (a) asserts that fj4.9|) holds. 
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